Training system and method for improving cognition and motor timing

ABSTRACT

A method of training a subject to improve anticipatory timing includes presenting the subject with multiple sequences of stimuli, receiving sensor signals associated with responses by the subject to the stimuli, and recording response times in accordance with the sensor signals, wherein the recorded response times are associated with the responses by the subject to the multiple sequences of stimuli, and wherein the response times have an associated distribution. The method further includes analyzing the distribution of response times for the subject to determine variability in anticipatory timing of the subject with respect to the stimuli, and generating a subject feedback signal corresponding to a result of the analyzing so as to train the subject to improve anticipatory timing of the subject. Also disclosed is a system for performing the aforementioned training method.

This application is a divisional patent application of U.S. patentapplication Ser. No. 10/834,356, “Cognition and Motor Timing Diagnosisand Training System and Method,” filed Apr. 27, 2004, now U.S. Pat. No.7,384,399 which is hereby incorporated by reference.

TECHNICAL FIELD

The disclosed embodiments relate generally to systems and methods oftesting a person's ability to anticipate stimuli, as well as remedialsystems and methods for improving a person's ability to accuratelyanticipate the timing of predictable events or stimuli.

BACKGROUND

Anticipation or expectation of a sensory event that must be paired withan action is essentially a form of attention that is crucial for anorganism's interaction with the external world. The accurate pairing ofsensation and action, assuming these functions are intact, is dependenton timing and is called sensory-motor timing, one aspect of which isanticipatory timing. Anticipatory timing is essential to successfuleveryday living, not only for actions but also for thinking. Thinking orcognition can be viewed as an abstract motor function and therefore alsoneeds accurate sensory-cognitive timing. Sensory-motor timing is thetiming related to the sensory and motor coordination of an organism wheninteracting with the external world. Anticipatory timing is usually acomponent of sensory-motor timing and is literally the ability topredict sensory information before the initiating stimulus.

Anticipatory timing is essential for reducing reaction times andimproving both movement and thought performance. Anticipatory timingonly applies to predictable sensory-motor or sensory-thought timedcoupling. The sensory modality (i.e., visual, auditory etc.), thelocation, and the time interval between stimuli, must all be predictable(i.e., constant, or consistent with a predictable pattern) to enableanticipation movement or thought.

Without reasonably accurate anticipatory timing, a person cannot catch aball, know when to step out of the way of a moving object (e.g.,negotiate a swinging door), get on an escalator, comprehend speech,concentrate on mental tasks or handle any of a large number of everydaytasks and challenges. This capacity for anticipatory timing can becomeimpaired with aging, alcohol, drugs, hypoxia, infection, clinicalneurological conditions including but not limited to Attention DeficitHyperactivity Disorder (ADHD), schizophrenia, autism and brain trauma(head injury or concussion). For example, brain trauma may significantlyimpact a person's cognition timing, one aspect of which is anticipatorytiming. Sometimes, a person may appear to physically recover quicklyfrom head or brain trauma, but have significant problems ofconcentration, memory, headaches, irritability and other symptoms as aresult of impaired anticipatory timing. In fact, impaired anticipatorytiming may cause the person to suffer further injuries by not having thetiming capabilities to avoid another accident.

SUMMARY OF DISCLOSED EMBODIMENTS

A method of training a subject to improve anticipatory timing includespresenting the subject with multiple sequences of stimuli, receivingsensor signals associated with responses by the subject to the stimuli,and recording response times in accordance with the sensor signals,wherein the recorded response times are associated with the responses bythe subject to the multiple sequences of stimuli, and wherein theresponse times have an associated distribution. The method furtherincludes analyzing the distribution of response times for the subject todetermine variability in anticipatory timing of the subject with respectto the stimuli, and generating a subject feedback signal correspondingto a result of the analyzing so as to train the subject to improveanticipatory timing of the subject. Also disclosed is a system forperforming the aforementioned training method.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a sequence of stimuli presented to a person, in thiscase two circles that alternately go on and off in a predictable orrandom fashion.

FIG. 2 a illustrates a subject's response timing to sequences ofstimuli, both for predictable and random stimuli.

FIG. 2 b shows normal and abnormal distributions of anticipatory timing.

FIG. 3 is a conceptual block diagram of a cognition timing diagnosis andtraining system.

FIG. 4 is a detailed block diagram of a cognition timing diagnosis andtraining system.

FIG. 5 is a flow chart of a cognition timing diagnosis method.

FIG. 6 is a flow chart of an anticipatory timing remedial trainingmethod.

Like reference numerals refer to corresponding parts throughout theseveral views of the drawings.

DESCRIPTION OF EMBODIMENTS

Since it is extremely difficult to measure thinking performance withouta movement and since a similar neural network is used for anticipatorytiming, cognition and motor timing are linked. Therefore diagnosis andtherapy can be performed for anticipatory timing difficulties in themotor and cognitive domains using motor reaction times and accuracy. Inparticular, both the reaction time and accuracy of a subject's movementscan be measured. As discussed below, these measurements can be used forboth diagnosis and therapy.

Anticipatory cognition and movement timing are controlled by essentiallythe same brain circuits. Variability or a deficit in anticipatory timingproduces imprecise movements and disrupted thinking, such as difficultyin concentration, memory recall, and carrying out both basic and complexcognitive tasks. Such variability and/or deficits leads to longerperiods of time to successfully complete tasks and also leads to moreinaccuracy in the performance of such tasks. Some embodiments of thepresent invention measure such variability or deficits to determinewhether a person suffers impaired anticipatory timing. Some embodimentsof the present invention use a sequence of stimuli in combination with afeedback mechanism to train a person to improve anticipatory timing.

Sequenced stimuli presented to a subject may include sequences of bothpredictable and non-predictable (e.g., random or pseudo-random) stimuli.In one embodiment, the non-predictable stimuli are presented to asubject before the predictable stimuli. The stimuli can use any sensorymodality. In some embodiments, the stimuli are visual stimuli. In otherembodiments, the stimuli are auditory. While other forms of stimuli canbe used, the embodiments described here use visual stimuli. Thesubject's responses may be visual, manual or even spoken. In someembodiments the subject's responses are measured by tracking eyemovement. In other embodiments, the subject's responses are measured bya mechanical, piezoelectric or other sensors activated by physicalmovement of the subject, such as pressing a button. In yet otherembodiments, a frontal brain electroencephalographic (EEG) signal (e.g.,the “contingent negative variation” signal) is measured during theperiod before a subject's response. The amplitude of the EEG signal isproportional to the degree of anticipation and will be disrupted whenthere are anticipatory timing deficits.

FIG. 1 depicts a sequence of visual stimuli. The subject's task is tofollow the illuminated circle, which will alternate between twolocations at a random (non-predictable) or non-random (predictable)rate. In one embodiment the random rate is between 500 msec to 2 sec.The subject may indicate that the circle is illuminated at a particularlocation by activating a sensor, or by moving his/her eyes to focus onthe illuminated circle on the screen. In another example, the subjectmay indicate that the circle is illuminated at a particular location bycontrolling the position of an image of an object on a screen using ajoystick or other user input mechanism, and may be asked to move theobject so as to “catch” a moving object, or to avoid being hit byanother object, or to move the object so as to match the movement of acomputer generated image, or other similar exercise.

In yet another example, multiple objects, such as images of circles, aredisplayed in a row or other pattern. The objects are flashed on and offin a predictable or random fashion. Eye movement reaction times aremeasured by a digital video infrared camera focused on the subject'spupil, operating at a picture update rate of at least 200 hertz. Theresulting digital video signal is analyzed by a computer to determinethe screen position(s) where the subject was focusing, and the timing ofwhen the subject focused at the appropriate screen position. If feedbackis provided, the feedback may be provided by giving the subject a tone,using either open air audio speakers or headphones, or by having thecolor of the display change when the subject's anticipatory timing iswithin a normal or desired distribution.

In some embodiments, the stimuli presented to the subject include one ormore sequences of non-predictable stimuli. The non-predictable stimulican be random or pseudorandom sequences. The sequences ofnon-predictable stimuli cannot be learned and there is therefore noanticipatory timing by the subject. Measurements of the timing of thesubject's responses to the sequences of non-predictable stimuli can beused as an internal control. These measurements are measurements of thesubject's reactive timing. Subtracting the subject's reactive timingfrom the subject's anticipatory timing produces the subject's absoluteanticipatory timing. By taking numerous timing measurements of thesubject's responses to sequences of non-predictable stimuli, adistribution of such timing measurements is generated. The distributioncan be graphed or displayed, compared with normative data for apopulation of other subjects, and the like.

Next, the stimuli presented to the subject also include multiplesequences of predictable stimuli. An initial phase in which thesequences of predictable stimuli are presented is called the learningphase. During the learning phase there is typically a progressive shifttoward earlier correct reactions, and thus anticipatory timing. It isnoted that in some embodiments, incorrect reactions by the subject areeliminated or not used for purposes of evaluating anticipatory timing.After the learning phase, there should be an anticipatory reaction phaseduring which the subject's response times are relatively static orfixed. The subject response times during the anticipatory reaction phasewill generally be earlier than the initial responses during the learningphase. These response times, herein called anticipatory timing, willalso be shorter than the subject's reactive timing to non-predictablestimuli.

By testing the subject with numerous sequences of predictable stimuliand taking measurements of the subject's anticipatory timing, adistribution of such timing is generated. The distribution can begraphed or displayed, compared with normative timing data for apopulation of other subjects, and the like.

FIG. 2 a is a prophetic example of a graph of typical response timingsto sequences of stimuli, both for predictable and random stimuli. Forpurposes of this discussion the terms “normal subject” and “abnormalsubject” are defined as follows. Normal subjects are typically healthindividuals whose sensory-motor or anticipatory timing falls within anormal performance range. Abnormal subjects are individuals sufferingfrom impaired brain function with respect to sensory-motor oranticipatory timing.

As represented in FIG. 2 a, even normal, health subjects responding torandom stimuli (♦) cannot anticipate the exact timing of the stimuli,and thus they lag behind being “on target.” In other words, even after alearning phase where the user is subjected to a number of sequences ofstimuli, the normal user cannot anticipate a subsequent sequence ofrandom stimuli.

Normal subjects responding to predictable stimuli (●), such as arepeating sequence of visual stimuli, after a learning phase start toanticipate the stimuli before they are presented to the subjects. Duringa learning phase the normal subjects learn the sequence of stimuli andare then able to anticipate the stimuli during an anticipatory phase.Abnormal subjects (▪), however, only slightly improve their responsetiming after the learning phase and still cannot anticipate the stimuliduring the anticipatory phase. In other words, abnormal subjects mayimprove their response timing during training, but cannot anticipatesubsequent stimuli as well as a typical normal subject.

FIG. 2 b is a prophetic example of the distribution of anticipatoryresponse timing of an abnormal subject and the average anticipatoryresponse timing of a control group of normal subjects. An abnormaldistribution of anticipatory response timing is typically slower, onaverage than the normal distribution. The abnormal subject alsotypically has more inaccurate responses. Even more significantly, thewidth of an abnormal anticipatory timing distribution is typicallysignificantly wider than the width of a normal anticipatory timingdistribution. In some embodiments, the width of a distribution may bedefined as the full width of the distribution at half maximum (sometimescalled FWHM). In some embodiments, the width of a subject's anticipatorytiming distribution is defined as the variance of the responsedistribution, the standard deviation of the response distribution, theaverage deviation of the response distribution, the coefficient ofvariation of the response distribution, or any other appropriatemeasurement of the width of the response distribution.

In some embodiments, as described above, the subject's reactive timingis subtracted from the subject's anticipatory timing to produce thesubject's absolute anticipatory timing. In some embodiments, this isaccomplished by subtracting an average reactive timing value from theanticipatory timing values.

The subject's absolute anticipatory timing distribution can be comparedwith the absolute anticipatory timing distribution of a control group ofsubjects. Both the average timing and the width of the timingdistribution, as well as their comparison with the same parameters for acontrol group are indicative of whether the subject is suffering from acognitive timing impairment.

FIG. 3 is a conceptual block diagram of a cognition timing diagnosis andtraining system 100. The system includes a computer 102 coupled one ormore actuators 104, and one or more sensors 106. When the system isconfigured for use as a cognitive timing training system, the system 100may also include one or more feedback devices 110. In some embodiments,feedback is provided to the subject via the actuators 104. The one ormore actuators 104 may include a display device for presenting visualstimuli to a subject, audio speakers for presenting audio stimuli, acombination of the aforementioned, or one or more other devices forproducing or presenting sequences of stimuli to a subject. The one ormore sensors 106, may be mechanical, electrical, electromechanical,auditory (e.g., microphone), visual sensors (e.g., a digital videocamera) or other type of sensors (e.g., a frontal brainelectroencephalograph, and known as an EEG). The job of the one or moresensors 106 is to detect responses by a subject to sequences of stimulipresented by the one or more actuators 102. Some types of sensorsproduce large amounts of raw data, only a small portion of which can beconsidered to be indicative of the user response. In such systems, thesensor or computer 102 contain appropriate filters and/or softwareprocedures for analyzing the raw data so as to extract “sensor signals”indicative of the subject's response to the stimuli. In embodiments inwhich the one or more sensors 106 includes an electroencephalograph(EEG), the relevant sensor signal from the EEG may be a particularcomponent of the signals produced by the EEG, such as the contingentnegative variation (CNV) signal or the readiness potential signal.

The one or more feedback devices 110 can be any device appropriate forproviding feedback to the subject. The one or more feedback devices 110provide real time performance information to the subject correspondingto measurement results, which enables the subject to try to improvehis/her anticipatory timing performance. In some embodiments, theperformance information provides positive feedback to the subject whenthe subject's responses, in response to sequences of stimuli, are withina normal range of values. In some embodiments, the one or more feedbackdevices 110 may activate the one or more actuators 104 in response topositive performance from the subject, such as by changing the color ofthe visual stimuli or changing the pitch or other characteristics of theaudio stimuli.

FIG. 4 is a block diagram of a cognition timing diagnosis and training(or remediation) system 400. The system 400 generally includes one ormore processors 402, such as CPUs, a user interface 404, memory 412, andone or more communication buses 414 for interconnecting thesecomponents. The system 400 may optionally include one or more network orother communications interfaces 410, such as a network interface forconveying testing or training results to another system or device. Theuser interface 404 includes at least one or more actuators 104 and oneor more sensors 106, and may also include one or more feedback devices110, as discussed above. In some embodiments, the user interface 404 mayfurther include additional computer interface devices such as a keyboardand/or mouse 405 and a display 406 (although the display may one of theactuators 104).

Memory 412 may include high speed random access memory and may alsoinclude non-volatile memory, such as one or more magnetic disk storagedevices. Memory 412 may include mass storage that is remotely locatedfrom the central processing unit(s) 402. The memory 412 stores anoperating system 416 (e.g., Microsoft Windows, Linux or Unix), anapplication module 420, and may optionally store a network communicationmodule 418. The application module 420 may include:

a stimuli generation control program, module or instructions 422, forgenerating sequences of stimuli, as described elsewhere in thisdocument;

an actuator or display control program, module or instructions 424, forproducing or presenting the sequences of stimuli to a subject;

a sensor control program, module or instructions 426, for receivingsensor signals and, where appropriate, analyzing raw data in the sensorsignals so as to extract sensor signals indicative of the subject'sresponse to the stimuli; the sensor control program, module orinstructions 426 may also include instructions for controlling operationof the one or more sensors 106;

a measurement analysis program, module or instructions 428, foranalyzing the sensor signals to produce measurements and analyses, asdiscussed elsewhere in this document; and

a feedback program, module or instructions 430, for generating feedbacksignals for presentation to the subject via the one or more actuators orfeedback devices.

The application module 420 may furthermore store subject data 432, whichincludes the measurement data for a subject, and optionally may alsoinclude analysis results and the like. The application module 420 mayalso store normative data 434, which includes measurement data from oneor more control groups of subjects, and optionally may also includeanalysis results, and the like, based on the measurement data from theone or more control groups.

Still referring to FIG. 4, in an exemplary embodiment, the one or moresensors 106 include a digital video camera focused on the subject'spupil, operating at a picture update rate of at least 200 hertz. In someembodiments the digital video camera is an infrared camera, while inother embodiments the camera may operate in other portions of theelectromagnetic spectrum. The resulting video signal is analyzed by theone or more CPU's 402, under the control of the measurement analysisprogram, module or instructions 428, to determine the screen position(s)where the subject focused, and the timing of when the subject focused atone or more predefined screen positions.

Referring to FIG. 5, in an embodiment of a method of analyzing asubject's ability to anticipate predictable stimuli, the subject ispresented with multiple sequences of stimuli, including sequences ofnon-predictable stimuli and sequences of predictable stimuli (502).Sensor signals associated with the subject responding to thenon-predictable stimuli are received (504). Similarly, sensor signalsassociated with the subject responding to the predictable stimuli arereceived (506). In some embodiments, sensor signals associated with thesubject responding to the predictable stimuli are received during aninitial learning phase (512) followed by an anticipatory phase (514), asdescribed above in relation to FIG. 2 a. In some embodiments, thelearning phase lasts five to ten sequences of stimuli (trials). Timingvalues associated with the sensor signals are recorded for a pluralityof sequences of stimuli (508). The recorded timing information isanalyzed to determine if the subject has an anticipatory timingimpairment (510). In some embodiments, a report is generated to presentthe analysis.

Referring to FIG. 6, in an embodiment of a method of training a subjectto improve his/her response times to predictable stimuli, the baselinetesting and analysis is performed to determine and/or analyze asubject's reactive timing, anticipatory timing and learning (602). Thismay correspond to a method similar to the one represented by the flowchart in FIG. 5.

Next, a sequence of training steps 604-612 are repeatedly performed soas to help train a subject to improve his/her anticipatory timing. Suchtraining exercises portions of the subject's brain that are responsiblefor cognitive tasks associated with anticipating events. By focusing thetraining narrowly on those cognitive tasks associated with anticipatingevents, appropriate portions of the brain are stimulated, which causesthe brain to find ways to improve the subject's ability to anticipatethe timing of predictable events.

In some embodiments, the training steps include presenting the personwith one or more sequences of predictable stimuli (604) and receivingsensor signals associated with the subject responding to or anticipatingthe predictable stimuli (606). The sequences of predictable stimuli mayinclude precursor stimuli and target stimuli, with the precursor stimuliproviding the subject the information needed to predict the targetstimuli. For example, the precursor stimuli may display an arrowpointing to the area in which a visual stimulus will shortly appear.Timing values associated with the sensor signals are determined (608)and compared with predefined criteria, such as baseline data for one ormore control subjects (610). Based on the results of the comparison, asubject feedback signal corresponding to the results may be generated(612). In some embodiments, only positive feedback signals aregenerated, to reward performance meeting predefined or individuallydetermined performance goals. In other embodiments, the feedback signalsinclude negative feedback signals that indicate failure to meet theperformance goals. In still other embodiments the feedback signals mayinclude gradations to indicate the extent to which the subject has metor failed to meet the performance goals.

The foregoing description, for purpose of explanation, has beendescribed with reference to specific embodiments. However, theillustrative discussions above are not intended to be exhaustive or tolimit the invention to the precise forms disclosed. Many modificationsand variations are possible in view of the above teachings. Theembodiments were chosen and described in order to best explain theprinciples of the invention and its practical applications, to therebyenable others skilled in the art to best utilize the invention andvarious embodiments with various modifications as are suited to theparticular use contemplated.

1. A system for training a subject to improve anticipatory timing, comprising: a display to present to the subject multiple sequences of visual stimuli; a sensor that produces sensor signals corresponding to responses by the subject to the visual stimuli; one or more processors; and memory storing one or more programs to be executed by the one or more processors, the one or more programs including instructions for performing a sequence of operations, including: presenting to the subject the multiple sequences of visual stimuli; recording response times in accordance with the sensor signals, wherein the recorded response times are associated with the responses by the subject to the multiple sequences of stimuli, and wherein the response times have an associated distribution; analyzing the distribution of response times for the subject to determine variability in anticipatory timing of the subject with respect to the stimuli; and presenting a subject feedback signal to the subject, the feedback signal corresponding to a result of the analyzing, and repeating the sequence of operations so as to train the subject to improve anticipatory timing and reduce variability of the subject responses with respect to the stimuli.
 2. The system of claim 1, including a feedback device for providing feedback to the subject in accordance with the subject feedback signal.
 3. The system of claim 1, wherein the instructions for analyzing include instructions for comparing the anticipatory timing of the subject with normative data and classifying the anticipatory timing of the subject, including the variability of the anticipatory timing of the subject, as normal or abnormal in accordance with a result of the analyzing.
 4. The system of claim 1, wherein the visual stimuli consists of a smoothly moving object, repeatedly moving over a tracking path and the sensor signals are signals derived from video signals monitoring eye movements of the subject; and wherein the determined variability corresponds to how consistently eye movements of the subject visually track movement of the object.
 5. The system of claim 4, wherein the sensor comprises a camera operating at a picture update rate of at least 200 Hertz.
 6. The system of claim 1, wherein the sensor is mechanical or piezoelectric sensor that responds to a manual action by the subject.
 7. The system of claim 1, wherein the instructions for presenting the subject feedback signal are responsive to repeated analysis of the subject's variability in anticipatory timing with respect to the visual stimuli. 